Along with the development of television technologies and multimedia processing technologies, people have an increasing demand for the most realistic visual experience, and three-dimension televisions are regarded as the next generation of products subsequent to color televisions and high-definition televisions. With the principle that a person observes an object with his or her eyes at slightly different angles and thus can discern how far the object is away and obtain a stereo vision, a three-dimension television separates images as seen by the left and right eyes from each other so that the user can experience a stereo perception with his or her naked eyes without any stereo glasses. Particularly multi-viewpoint videos are ascertained to have a very wide application prospect, and virtual viewpoint synthesis in the multi-viewpoint videos is a key technology.
At present, depth image based rendering is a method for synthesizing virtual viewpoints using a reference viewpoint image and corresponding depth information, and this method has gained increasing attention and applications in recent years due to its good encoding efficiency, two-dimension/three-dimension selectivity and user interactivity. However, this method still suffers from the following drawback: an area which shall be visible at a virtual viewpoint is obscured at an original viewpoint, that is, an object or a background present in a virtual viewpoint image (hereinafter an area where the object or the background is present is referred to as an exposed area) is absent at a corresponding location in an original image, and information about the exposed area is absent in the original image, so the exposed area is a hole. The hole is embodied by a boundary with a sudden change in gray scale value in a depth image, so the hole has to be filled if the image quality of the virtual viewpoint needs to be improved.
At present, in order to improve the image quality of the virtual viewpoint and generate a resulting image with an optimum stereo effect, in a general approach to address the problem of a hole, depth image preprocessing is performed on the original image before three-dimension image transformation is performed on the original image to make a sharp sudden change in grade scale value in the depth image of the original image become smooth and further alleviate the difference in depth between a foreground object and a background to thereby lower the number of holes in the virtual viewpoint image and improve the quality of the view. Although this approach improves the quality of the virtual viewpoint image to some extent, depth image preprocessing may result in unrecoverable serious geometrical distortion of the virtual viewpoint image.